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Abstract 

For a set X, a set system CC2 X with finite VC dimension is maximum 
if it has the largest size allowable by Sauer's Lemma. There is a natural 
association between set systems and parameterized formulas in first order 
model theory, where the set system associated to a formula is known as a 
"definable family." Merging the two points of view, we may consider set 
systems which are maximum in the sense of computational learning theory, 
and stable in the sense of model theory. We show that all stable maximum 
families are of the form C C {AAB : A € [X]- n } for some n € to and B C 
X. We also examine maximum and non-maximum semi-algebraic families, 
give a model-completeness result generalizing the model completeness of 
(Q, <), and demonstrate that maximum families have UDTFS. 



1 Introduction 

This paper is about set systems, in particular those arising as definable families in 
certain formal structures. By "family," we mean a collection of similar objects, 
such as circles, triangles, or other concepts, defined and parameterized by a single 
first-order formula. The "similarity" of objects in a given family (or set system) 
is captured by its Vapnik-Chervonenkis (VC) dimension, defined below. Only set 
systems with finite VC dimension (so-called VC classes) will be considered. 

In this paper we explore the model-theoretic properties of definable families 
which are maximum and/or maximal. These terms refer to the largest set systems 
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of a given VC dimension, where "largest" is measured in two distinct ways. The 
maximal classes are apparently somewhat ad hoc, but maximum classes have a 
smoothness which gives strong structural properties. These objects have long 
been of interest to researchers working in the field of Computational Learning 
Theory (CLT), where they have applications to PAC learning^ 

The paper is organized as follows. In Section \T\ we give the main definitions, 
and explore the properties of set systems in the absence of any syntax or seman- 
tics. Section |2] introduces model theory, and gives some of the main results, such 
as Propo sition 12 . 8 1 and Theorem 12.151 Section |3] gives a theorem of Floyd, which 
shows that linearly parameterized semi-algebraic sets are maximum. We then 
demonstrate that not all semi-algebraic families are as nicely behaved. In Sec- 
tion ID we give a model completeness condition related to the maximum property, 
which generalizes the model completeness of (Q, <). Section [5] defines UDTFS 
and shows its relation to the "compression scheme" notion from CLT. We then 
illustrate how a compression scheme can be translated into a first-order condition, 
namely UDTFS. In this section we also state some open problems, and prove the 
equivalence of two of them. 

Most stability-theoretic definitions are from Shelah [fT4l . The Vapnik-Chervonenkis 
dimension was defined in [fT5l . A non-logician who wishes to read this paper 
might consult chapters 1,2 and 5 of [9], and chapters 1 and 2 of lfl4ll . 

1.1 

A formula ip(vi, v%, . . . , Vf.) is said to be partitioned if the free variables Vi, . . . ,Vk 
are partitioned into parts x = v ^ , . . . , v im and y = v ix , . . . , Vj k _ m . In this case we 
write <p(vi, v 2 , . . . , u&) = (p(x; y). Recall that as an ordinal, n = {0, 1, . . . ,n — 1}. 

Definition 1.1. For any partitioned formula (p(x; y), n E u, and 77 : n — > 2, define 

(Pr,{yx, ■ ■ ■ , y n ) = 3x f\ ien (p(x; yi) v{l \ where (p(x; y) 1 := tp(x; y) and (p(x; yf := 

Definition 1.2. For any partitioned formula ip(x; y), define A„ >n = {ip v (yi, ■ ■ ■ , y n ) ■ 
7] : n — > 2}. 

Definition 1.3. Say that a sequence {ai) i&1 is A ^ n -indiscernible if for any i\ < 

■ ■ ■ < i n andji < • • • < j n from I, tp A ^ n (a il} . .. } a in )= tp^ n (a h , . . . , a jn ). 

Definition 1.4. Let X a set, A C X, and C C 2 X . Define C(A) = {cf]A : c e C}. 
Say that C shatters A if C(A) = 2 A . Let the VC dimension of C, denoted VC(C), 
be defined as sup{|v4| : A C X, C shatters A}. 

l A reader wishing to review this literature might first read E] and then skim BlfTllTl fTTIl . 
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For n G u, d G u, define $ d (n) = £)? =0 (™) . 

Lemma 1.5 (Sauer's Lemma [Q2l EES [13). Suppose C C 2 X /or a sef X. // 

VC(C) = d, and A C X is finite, then 

\C(A)\ < $ d (\A\) 

Definition 1.6 (El). Suppose C C 2 X and VC(C) = d. Say that C is maximum 
of VC-dimension d (or d-maximum) if for all finite A C X, 

\C(A)\ = U\A\) 

Definition 1.7 (O). Suppose C C 2 X and VC(C) = d. Say that C is maximal of 
VC-dimension d (or d-maximal) if for any ce2 x \C, VC(C U {c}) = d + 1. 

Example 1.8. C = [X]- d for any X with |X| > d is d-maximal, where [X]- d := 
{^4 C X : | A | < d}. It is also d-maximum. 

Example 1.9. Let X = {0,1,2,3}, d = 2, and build a suitable C. It is not 
hard to construct examples which are maximum and maximal, or maximal but not 
maximum. 

Any C is easily seen to have a maximal (though perhaps not a maximum) 
superclass 21 D C, with VC(2l) =VC(C), by Zom's Lemma G). 

Proposition 1.10. Le? C C 2 X for a set X and suppose C(A) is d-maximal for 
every finite A C X. Let 21 oe any d-maximal superclass of C. Then 21 is the 
closure ofC in the Tychonoff topology on 2 X , and in particular is unique. 

Proof Let X C X finite and / : X -> 2. We regard 2 X as x 2 and take sets 
of the form : X 2 | <j D /} as a basis for the topology. Let C be the 
closure of C. We must show C = 21. Let / G 21. For any finite X C X, let 
C /iXo = {g | a D / \ X }. Since C(Xo) is d-maximal, / \ X G C(X ). Then 
C/,x H C 7^ 0. This shows / G C It is easy to see that since 21 is maximal, it 
must be closed. Therefore 21 = C. □ 

Corollary 1.11. Let C C 2 X /or a sef X and suppose C is d-maximum. Then C is 
the unique d-maximal superclass ofC. 

That d-maximum classes have unique d-maximal superclasses was first shown 
in Floyd's thesis Q. That maximal classes are closed in the Tychonoff topology 
was observed by Dudley [0. 
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Given a partitioned formula (p(x; y), let <p*(y; x) = (p(x; y). For a monster model 
M, model N and B C M^, let C^(£)^ = M-B,6) : 6 G A^ 1 }. We let (^(E) 
where no model is specified implicitly denote C^[B) M . The shorthand C V (JA) 
will be used for C^(M^) M . 

Definition 2.1. Say that a partitioned formula <yj(x; y) is d-maximum (maximal) 
in M. if Ctp(JA) is d-maximum (maximal). Say that a partitioned formula is d- 
*maximum (maximal) in M. if C V *(JA) is d-maximum (maximal). 

Whether a formula is maximum depends only on the theory, since maximum- 
ness can be expressed as a set of first order sentences. For maximal families, 
on the other hand, consider the {<}-formula ip(x; y, z, w) which expresses the 
relation 

y = z A z < w 
y = z A z > w 

(1) 

y < z 
y > z 

We claim that this formula is 1 -maximal in R but not in Q. Note that the 
formula encodes all left cuts x < y including the "limit cuts" x < y, x ^ x, 
and x = x. The associated family is not maximal in Q because C ip (Q) Q does 
not include irrational cuts (which clearly do not increase the VC dimension). It is 
maximal in IR, since that structure is Dedekind complete. More precisely, (^(R) 18 
is closed in (ie. equals) C V (R). The claim then follows by Proposition ! 1.101 

Since <^(x; y, z, w) is also maximum, this shows that maximum does not imply 
maximal on an infinite domain (see also (US). On an y finite domain, however, 
maximum is easily seen to imply maximal. If F is a field and k C F is a proper 
subfield, then the formula (p(x; yi,...,y d ) = ((x - yi){x - y 2 ) • ■ ■ {x - y d ) = 
0) gives C^kY <i-maximum and d-maximal. However C v (¥) is only maximum 
(because it is "missing" the empty set). 



' x < y 


if 


x < y 


if 




if 


^ ilC ^ iC 


if 



Example 2.2. Let cp(x; yi, . . . , 7/ 2 m) be the L = {<} formula 

12,-1 < X < I)-,,) 



\J(V2 
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for some m G to. Then in any infinite linear order, cp is maximum of VC dimension 
2ml 



2 Unions of intervals have long been known to be maximum [3 1. An inductive proof is straight- 
forward. 
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Let C C 2 X be (i-maximum. For any A C I with |A| = d + 1, |C(A)| = 
+ 1) = 2 d+1 - 1. Let the unique A* e 2 A \ C(A) be called the forbidden 
label for C on A (Floyd's thesis, section 3.4). 

Example 2.3. Let X an infinite set, dGw and C = Then for any A C X of 
cardinality the forbidden label for C on A is A* = A. 

Example 2.4. Let X = Q and C = C x<y (Q). Then for {a, b} C <Q> with a < 6, 
the forbidden label for C on {a, 6} is {6}. 

When X has an ordering, a forbidden label can be naturally represented by a 
length d+1 binary string. In the first example, above, the missing label can be 

d+1 

viewed as 111 ■ • • 11, and in the second as 01. We can connect forbidden labels to 
model theory as follows. 

Proposition 2.5. Suppose that <p(x;y) is a d-maximum formula. Then for any 
a , • • • , a d G M^l, there is a unique vf : d+1 — > 2 such that |= ->(p v * (a , . . . , a^) A 
Ar/^r/* <A?(^cb • • • ) dd)- Moreover {aj : rj*(j) = 1} is the forbidden label of 
C V (M) on {a , . . .,d d }. 

We will call if in Proposition l2.5l the forbidden label of cp(x; y) on a , . . . , a d . 

Proposition 2.6 (Floyd (3]|). Suppose CC2 1 is d-maximum and d-maximal. For 
each A 6 [X] a!+1 let A* denote the forbidden label for C on A. Then for any 

ce2 x ,ceC ^ \/A e [X] d+1 (c n A ^ A*). 

Proof. Left to right is obvious. Right to left follows from the fact that C is d- 
maximal - any c satisfying the right hand condition cannot increase the VC di- 
mension of C, and is therefore already in C. □ 

By Sauer's Lemma, the hypothesis on C in Proposition 12 . 61 will hold whenever 
X is finite and C is d-maximum. 

This has an interesting consequence for maximum formulas. The following 
definition is a variation on classical NFCP from Keisler, which considers only 
positive instances. 

Definition 2.7. Say that a partitioned formula tp(x; y) is n-NFCP if for any B C 

M^l, any set {ip(x; d)* 1 ^ : a G B} of ±(/)-instances is consistent iff any T C 
{ip(x; a) v ^ a ' : a G B} with |T| < n is consistent. 

Proposition 2.8. Suppose the partitioned formula tp(x; y) is d-*maximum. Then 
<p(x;y) is (d+l)-NFCP. 
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Proof. Let p(x) = {(f(x; a)^ : a G B} be a set of ±<£>-instances, and sup- 
pose it is (d + 1) -consistent. By compactness we may assume B is finite. Let 
pos(p) = {a G B : 77(a) = 1}. Note that since B is finite, C^*{B) is maximal 
and maximum. By Proposition I2.6L pos(p) G C^^B), since it does not induce a 
forbidden label, by (d + 1) -consistency. But this implies that p(x) has a witness 
in Ai, and so is consistent. □ 

Proposition 12.81 can be juxtaposed with Helly's Theorem from combinatorial 
geometry (see ifTOl ). 

For any two functions 77 : n — > 2 and 7/ : m — > 2 on natural numbers m < n 
say that 7/ C 77 if there is an order preserving function v : to — )■ n such that for all 

1 G m, 7/(2) = f](u(i)). 

Theorem 2.9. Le? (p(x;y) be a d-*maximum formula. Suppose (ai) i€l is a se- 
quence compatible with y which is A v ^-indiscernible. Then (di) i( zj is A v jW - 
indiscernible. 

Proof. Suppose n G to, and that i x < • ■ ■ < i n , and ji < • • • , j n are subsequences 
of /. We must show tp Alp n (a h , . . . , a in ) = tp Avn (a h , . . . , a jn ), or equivalently 
that |= (fnia^, . . . , a in ) = (p n iflhi ■ ■ ■> f° r all 77 : n — >> 2. Let J*i = {77 : n — )• 

2 :(= (pr,(a h , . . .,a in )} and J* 2 = {77 : n ->• 2 :(= ^(d^, . . .,%„)} 

Since (ai)ie/ i s A^ + i-indiscernible, all length d + 1 subsequences have the 



same forbidden label 77*, where 77* is as in Proposition l2.5l By Proposition [Z6l for 
any t] : n — > 2, the following are equivalent. 

1. T) G JF t 

2. not T]* □ 77. 

3. G J" 2 

Therefore tp Aip n (a h , . . . , d in ) = tp^ in {a h , . . . , a in ), and (a;) ie/ is A^ in- 
discernible. □ 



The following corollary to Theorem 12.91 says that maximum formulas allow a 
strong form of "extraction" of indiscernibles. 

Corollary 2.10. Let (p(x; y) be a d-*maximum formula and suppose (ai) i( zj is a 
sequence compatible with y. Then (fli)i & i contains a ^-indiscernible subse- 
quence. 

Proof. By Ramsey's Theorem, (di)i e i contains a A v d+i-indiscernible subsequence. 
By Theorem |2.9[ this sequence is also A^-indiscernible. □ 
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Definition 2.11 (Shelah). A formula Lp(x;y) is stable in M. if for some N £ 
u> there are not sequences Si, . . . , dj^ £ M^' and bi, . . . ,b N £ M^l such that 

ipia^bj^ -i=^ i < j. 

Lemma 2.12 (Shelah). Suppose <p(x; y) is stable, and (di) ie i is an infinite A ViTl - 
indiscernible sequence for n £ a;. TTzen (oj)iej w « ^^-indiscernible set. 

Proof. Suppose not. Then since every permutation is a product of transpositions, 
for some subsequence a io , . . . , a iv , . . . , Oj n _ 1 in (ai) ieI and some 77 : n — >• 2, 

Suppose without loss that 7/(7) = and 77(7 + 1) = 1. Define 

V>(*)= A V^cn^ 

and 5 = ^(.M). Note 

^(a io , . . . , a i; , a ij+1 , . . . , a iri _J = 3x £ 5 (-><p(z; a*,) A ip(x; a k+1 )) 

Fix iV £ to. Since (a^ier is a A^-indiscernible sequence, we may assume 
without loss that there are d'j in (ai) i€l , j = 1, 2, . . . , N such that < a[ < 
. . . < a' N < a il+1 . Also since (ai) i€l is a A^ n -indiscernible sequence, we have 

|= 3x £ B (-«p(x; a-) A <^(x; a^)) •<=>- i < j 

We can therefore find appropriate &i, . . . , in S so that Oj-)^ < ^ == ^ > ? < 7- 
Since iV was arbitrary, ip(x; y) is unstable, a contradiction. □ 

In the following, we let ciAc 2 denote the symmetric difference of sets c\ and 

c 2 - 

Lemma 2.13. Let <p(x; y) be a stable d-*maximum formula. Then there is a num- 
ber N £ cu such that for all ci, c 2 £ C v * (M), |ciAc2| < N. 

Proof. If d = then \C V * (JA) \ = 1 and the lemma is trivial, so assume d > 1. 

By compactness, it is enough to prove that for all ci, c 2 £ (^-(.M), |ciAc 2 | < 
K . Suppose, by way of contradiction, that for some ci, c 2 £ C^,* (Ad), D = ciAc 2 
is infinite. Without loss, assume D C cj. By Corollary 12. 101 and Lemma 12.1 2[ 
there is D' C D, an infinite A^ ^-indiscernible set. Let d , . . . ,a d be distinct 
elements in D'. Consider tp^ d+1 (a , . . . , a d ) . By Proposition [23J there is exactly 
one 77* : d + 1 — > 2 such that |= -i(pn* (do, . . . , a^). By choice of D, we know that 
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rj* is not a constant function. Suppose, without loss, that r/*(0) ^ T)*(l). Define 
ix : d + 1 — >• 2 as 



Since -D' is an indiscernible set, the following are equivalent. 

1. |= (pr,*(a , a x , ...,a d ) 

2. \= (p v *(ax,a , ...,a d ) 

3. |= ip^(d Q ,ax, . . . ,d d ) 

But then r]* is not the unique forbidden label for {a , . . . , d d }, contradicting Propo- 
sition [231 □ 

IfC C 2 X for a set X and c G C, define CAc = {/Ac : / G C}. This 
operation clearly preserves many properties. 

Proposition 2.14. LetC C 2 X for a set X. Then C is (maximum, maximal, stable) 
if and only ifCAA is (maximum, maximal, stable) for any A C X. 

If also 21 C 2 X and CAc = 21 then C = 2lAc, (because symmetric difference 
is associative and therefore C AcAc = C). Say that C is a subfamily of 21 if C C 21. 

Theorem 2.15. A d-*maximum partitioned formula ip(x; y) is stable iffC ip *(Ai) 
is a subfamily of [M^]- n Acfor some n G to and any c G C v * {M). 

Proof. Let <p(x\y) be a d-*maximum partitioned formula. Suppose cp(x; y) is 
stable. Then by Lemma I2T31 for any c G C^(A1), <V(jM)Ac C [A#!]- n for 
some n E to. Therefore C ip *{M) C [M^I]- n Ac, by associativity of symmetric 
difference. 

Now, conversely, suppose C tp *(M) C [_M' s ']- n Ac. By Proposition 12. 14L 
[.M' s ']- n Ac is a stable family of sets, and therefore C v * (M) is also. Then <p(x; y) 
is stable. □ 

In traditional notation, if C is a maximum stable family on a set X, then C C 
[X]^"AA for some ACIandnGw. 




?f(z) if z G (d -I 

1 — r)*(i) otherwise 
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Note that in any field, the formula <p(x\ y) = (p(x; y) = 0) where p is a polyno- 
mial with coefficients y will be stable and maximum. On the other hand, polyno- 
mial equalities in dimensions greater than one will still be stable, but not maxi- 
mum in general^] For instance, the symmetric difference of two distinct lines in 
R 2 is infinite. Surprisingly, some semi-algebraic families are still in some sense 
approximately maximum. 

For a set H of real valued functions on a set X and a real valued function / (x) 
on X, let f -H= {f (x) - f(x) : f 6 H}. Let pos(f) = {x e X : f(x) > 0}. 
Define pos(f - H) = {pos(f (x) - f(x)) : / G H}. 

Proposition 3.1 (Floyd 0, Theorem 8.2). Let Rhea n-dimensional vector space 
of real valued functions on the set X, such that for every X e [X] n , H restricted 
to X is also n-dimensional. Further, for the real valued function fo(x) on X, 
assume that there are at most n elements of X such that fo(x) — f(x) = Ofor any 
f G H. Then the class C = pos(fo — H) is a maximum class ofVC dimension n 
on X. 

Proposition 13.11 builds on the theorem by Dudley [2] (Theorem 4.2.1) that 
families of the form pos(fo — H) have VC dimension dim(H). 

Example 3.2 (Floyd [3], p. 104.). Let X be a subset of M 2 , let H be the three- 
dimensional vector space of functions of the form f((x,y)) = a 3 y + a 2 x + ai, and 
let/ ((a:,y)) = -x 2 -y 2 . Then f ((x, y))-f((x, y)) = -x 2 -y 2 -a 3 y-a 2 x-ai, 
and pos(f — /) consists of all points for which x 2 + y 2 + a 3 y + a 2 x + ai < 
0. These are the points contained in the circle with center (— a 2 /2, — 03/2), and 
with radius ^(a 3 /2) 2 + (a 2 /2) 2 - a 1 . Restrict X to a subset of R 2 such that H 
is a 3-dimensional vector space on every subset of X of cardinality 3. This is 
satisfied if, for every three points (xi, yi), (x 2 , 2/2), (^3, 2/3) in X, the three vectors 
(xi, x 2 , x 3 ), (y\, y 2 , y 3 ), and (1,1,1) are linearly independent. Thus, X cannot 
contain 3 collinear points. Further restrict X to a subset of M 2 such that at most 3 
points lie on the circumference of any circle. Then pos(f — H) is a 3-maximum 
class on X. 

The X in the above example can be taken as dense in an extension of M 2 . 
In particular, add countably many new constants dj, paired into countably many 
2-tuples di and let T in the language of ordered rings (with constants from R) 
express 

1 . The di are dense in the order topology. 

3 To see that polynomial equalities are stable formulas, note that they are quantifier free, and 
that every field is contained in its algebraic closure (which is stable). 



9 



2. The <2j satisfy the conditions described in Example 13 .21 

Then this consistent set of sentences will give X as desired. 

The linear behavior of the parameters in Example 13 .2 l is important. 

Proposition 3.3. Let (p(xi, x<i\ z%, z 2 , z 3 ) = x\ + x\ + 23X2 + -22^1 + %\ < as 
in Example 13.21 There is no dense Y C M 3 , for any R -< Ai so that C v * (Y) is 
maximum. 

Proof. The reader may check that VC(y?*)=2. Suppose by way of contradiction 
that there is an extension Ai of R and a set Y C M 3 , dense in the order topology, 
so that C V *(Y) is 2-maximum. Let Ai, A 2 , A 3 , and A A be subsets of the plane 
so that Aj n Ai n A t ^ for any i,j, and /, but A 1 n A 2 n A 3 n A 4 = 0. We 
may assume without loss that for each i = 1, 2, 3, 4 there is a finite set Pi(x) of 
±-instances of yjfx; z) over K such that Ai = pi(R 2 ). Then the set of formulas 
E(x) = pi U p 2 U P3 U P4 is 3-consistent. By Proposition I2.8L is consistent. 
Since E is finite, and without loss finitely realizable in IR, it is realized in R. But 
this contradicts the fact that the have A 1 n A 2 n A 3 n A 4 = 0. □ 

Thus linearly parameterized semi- algebraic families are maximum on sets in 
"general position," but those with non-linear parameterizations can strongly fail 
to be maximum. 

Note that if we make a class C from the topological "frontiers" of the objects 
described in Example 13.21 then we get a stable maximum family of the type de- 
scribed in Theorem 12. 151 



Recall that a theory T is model complete if whenever Ai , N |= T and Af C /A, 
it follows Af Ai. The model completeness of (Q, <) can be generalized as 
follows. 

Proposition 4.1. Fix d e to U {00}. We will show that if L = {R(x,y)} is a 
language with a single binary relation, and T a L-theory with the axioms 

1. R(x, y) d-maximum 

2. R(x, y) is symmetric (or antisymmetric) 

3. No finite R(x, y)-type is algebraic, 
then T is model complete. 
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Proof. We use Robinson's test. Suppose J\f C M. and Af,M \= T. Consider the 
q.f. formula 

a(v,v) = f\(vi = v)* 1 ® A f\ R{v h vf 2{i) A f\ R(v, Vi ) m{l) 

Vidv Vi£v Vj£v 

where rjj : \v\ — > 2, for all j G {1, 2, 3}. 

Let a G iV^I, and suppose M. |= 3t>cr(a,t>). We must show M |= a(a,6) 
for some b E N. Without loss of generality, we may assume that r/i is the zero 
function. We first show that 

J\f \=3v f\ R(a u V y 2(i) A f\ R{v, 

Since R(x, y) is assumed to be symmetric (or antisymmetric) and a is consis- 
tent, the above statement holds if and only if 

N |= 3v /\ R{v,a t ) r ' 2{i) (2) 

But this last statement must hold. For if not, 

\CR{range{a)) Jsf \ < \Cn(range(a)) M \ 
contradicting that R is maximum in J\f. 

By condition 3 and equation (1), J\f has a witness b to 3v /\ e - R(v, ai) m ® 
such that 

l\{a^b) 

Thus 

Af \= o-M) 

and consequently T is model complete. □ 

If (2) in Proposition 14.11 is replaced with the assumption that R(x, y) is 2- 
sorted, or that x and y are otherwise incompatible, then an analog of the proposi- 
tion goes through, if we further assume that R*(y, x) is maximum. 
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5 UDTFS 



If B C M^l and 92(0;; y) is a partitioned formula, a complete (/?-type over £? is any 
consistent set of formulas 

p{x) = {ip{x-ly® :beB} 

for some rj : B — > 2. We let S^B) represent the set of all complete 93-types over 
B. 

Definition 5.1. Let (p(x,y) a partitioned formula. We say that ip has uniformly 
definable types over finite sets (UDTFS) if for some N E to there exists a set of 
formulas {ipi(y, yo, ■ ■ ■ , y n -i) '■ I £ N} such that for any any finite B C and 
any p G S^B) there is / G iV and 6 0; • • • , b n -i G £? such that for all b E B, 

<f(x,b)ep <=^\=i/)i(b,b ,...,b n -i). 

The definition of UDTFS was based on the notion of a compression scheme 
from computational learning theory |6|. Warmuth and Little stone flU say that C 
admits a d-dimensional compression if, given any finite subset F of X, and any 
set A G C, there is a rf-element subset S of F such that the set A fl F can be 
recovered from the sets S (1 A and £ \ A. 

Example 5.2 (Warmuth & Littlestone). Let C be the set of all solid axis-parallel 
rectangles in the plane and F a finite set of points. Fix a rectangle R G C. Let S 
be the topmost,leftmost,rightmost and lowest points in F fl R. Let R = f]{R' G 
C : S C R'}. Then R(~) F = R(~) F. 

Note that in Example 15 .2[ S \ R was not needed, and it is not necessary that 
R G C. 

A more technically useful tool is an extended compression scheme, also due 
to Warmuth and Littlestone, and defined as follows. 

In the following, we identify C C 2 X with {f c : X -> 2 : c G C}, where 
/ c (x) = 1 <^=^> x G c. Furthermore, a function is identified with its graph, so 
that fi C / 2 iff /1 is a restriction of / 2 . 

For B C X, the notation C|b denotes the set of restrictions {f\s ■ f G C} and 

C| fin = |J{C|b : B a finite subset of X with |S| > 2} 

Definition 5.3. Fix C C x {0, 1}. C is said to have an extended d-compression if 
there is a compression function k : C\& n — > [X]- d and a finite set 1Z of reconstruc- 
tion functions p : [X]- d — >■ x {0, 1} such that for every / G C| fin 

1. K(f)Cdom(f) 
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2. / C for at least one p ElZ. 

We say that C has an extended d-sequence compression if there there is a com- 
pression function k : C|fi n — >■ X d and a finite set 1Z of reconstruction functions 
p : X d — > x {0, 1} such that for every / G C|fi n , range(/t(/)) C dom(f), and 
/ ^ p( K (f)) f° r at l east one p G 7£. 

The existence of either of these ^-compressions is equivalent. See [[6] for 
proofs. 

Theorem 5.4 (01, Theorem 11). Suppose C C 2 X for a set X is d-maximum. 
Then C has a d-dimensional compression. 

This was later improved to an extended ^-compression with \1Z\ = 1 in fTl. 
An extended ^-compression with \1Z\ = 1 is usually called an "unlabeled" d- 
compression scheme. While the improved result does not translate to a first-order 
statement (ie UDTFS) the original result does, as we now show. 

Definition 5.5. Let cp(x; y) a partitioned formula and B C finite. Say that 
p E Sip(B) internally shatters A EL Blip \ b \a has 2' A ' extensions to B. 

Define the independence dimension of cp(x; y) as VC(y2*), and say that A C 
M^l is independent if A is shattered by (p*(y; x). 

Lemma 5.6. Suppose f(x; y) is a partitioned formula with independence dimen- 
sion d. Suppose further that for any finite B C and p E S V (B), p internally 
shatters some A E [B] d . Then ip(x;y) is UDTFS. 

Proof. Let B, p and A be given. Let b G B \ A and consider ip(x; ft)*. Since 
\A\ = d, there must be exactly one value for t G 2 for which ip{x] tif is consistent 
with any q G S V (A), namely the value for which (p(x; &)* G p. Otherwise A U {b} 
would be independent, contradicting Idim(<p) = d. Define 

6{y;y ,...,y d -i) = /\ 3x I (p(x\ y) A /\ (p(x; yi) v{i) J 

For i] : d — > 2, define 



l\{y = yi^{y = yf {i) )^ \/\y^Vij^ Km Vo,---, m-i) 

i£d Vied / 



Now for any p G S^B), if a±, . . . , is internally shattered by p, then for 
some 77 : d — >■ 2 and all b E B, 

(f(x, b) Ep (= ^,(6, Si, ••• , S d ). 

The formulas ^(i/i 2/o, • • • , suffice. □ 
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The fact that all d-* maximum formulas satisfy the hypothesis in Lemma [5761 
is attributed to Emo Welzl ffTTl . See also Theorem 10 in 0). 

Lemma 5.7 (Welzl, 1987). Suppose ip(x;y) is a d-*maximum partitioned for- 
mula. Then for any finite B C M' 5 ' and p G S^B), p internally shatters some 
A G [B] d . 

Theorem 5.8. Suppose ip(x] y) is a d-*maximum partitioned formula. Then <p(x; y) 
is UDTFS. 

Proof. By Lemmas l5.6l and l5.7[ □ 

The following are open questions regarding UDTFS. In each case the left to 
right direction is known to hold. A L-theory T has UDTFS if every L-formula 
has UDTFS. 

1. (p(x; y) is UDTFS iff C ip *(M) has an extended d-compression scheme for 
some d. 

2. Every reduct of a theory T has UDTFS iff T has UDTFS. 

3. (p(x; y) has UDTFS iff ip(x; y) has finite VC dimension. 

4. C C 2 X has a compression scheme of order d iff C has finite VC dimension 
d. 

Question (3) is the NIP -<=>- UDTFS conjecture discussed in [5j . Question 
(4) is one of the principal unsolved problems in computational learning theory 

una. 

Proposition 5.9. (1) (2) 

Proof. Let (p(x; y) be given and suppose (1) holds, viz that every cp(x; y) for which 
C V *(JA) has an extended compression scheme is UDTFS. Suppose T is an L- 
theory in which cp(x;y) is UDTFS. Let T' be a reduct of T to V C L such that 
(p G L'. We will show that (p(x;y) is UDTFS in T'. It is easy to see that (p(x;y) 
UDTFS in T implies that when M. (= T, C (p *(Ai) has an extended compression 
scheme. Let M' be an L' reduct of M. Then C V *(M) = C^^M'), and so 
C^* {M!) has an extended compression scheme. But then by (1), (p(x; y) is UDTFS 
in V. 

Now conversely suppose that (2) holds, viz that if T is UDTFS in L then the V 
reduct T' is UDTFS. Let ip{x] y) be given and suppose C lp *(M) has an extended 
compression scheme for an L model M. of T. We must show that ip(x; y) is 
UDTFS in T. Without loss of generality, C v * (At) has an extended sequence com- 
pression scheme, with reconstruction functions 1Z. For each p(y~o, . . . , Vd-i) € H, 
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add a predicate p'(y; y , . . . , yd-i) to L to get a language L n , Let .M 7 * be an L 7 ^ 
expansion of .M such that for all a, a , . . . , a rf _i G M n ^, 

p'(a,ao,...,a d -i) Mli p(a , . . . , 0,4-1) (a) = 1 

Then p(z; y) is UDTFS in Th(M n ). By (2), it is also UDTFS in the L reduct, 
T. □ 

Clearly if either (1) or (2) holds, then (3) and (4) are qualitatively the same 
question. 



6 Conclusions 

The following are typical examples of maximum VC classes: 

1. [xp 

2. Unions of boundedly many intervals in R 

3. Sets of positivity for a finite dimensional real vector space of real valued 
functions, restricted to points in general position. 

Note that the second example is a special case of the third. We have shown that 
all stable maximum families are essentially of the first type. It is unclear whether 
there are similar "universal" set systems for unstable maximum families, or if the 
above unstable examples are the only possibilities. Some work on this problem is 
done in [1J. Results on geometric characterizations of maximum families can be 
found in [[mi . as well as an algorithm for generating all finite maximum families. 

It has been remarked that there is a curious absence of natural examples of 
maximal but not maximum classes 0. At the same time, it seems that "most" of 
the wild (ie. random) maximal classes are not maximum [SID. 

On this same topic, the property of being maximal (in particular, closed in 
the Tychonoff topology) implies a strong condition on the type space which will 
not always be possible. A model Af with a binary relation R whose type space 
Cr(N)^ is closed in (ie equal to) Cr{N) must be more than saturated-it must re- 
alize all types over itself. Therefore, no equivalence relation with infinitely many 
classes will ever be closed (because of the "not equivalent to anything" type). In 
fact a Dedekind complete order is a rare example of something natural which is 
maximal. The notion of a closed relation is similar to algebraic compactness in the 
theory of modules, where a pure injective module must realize all positive prim- 
itive formulas over itself. That condition is relatively easy to realize, however, 
since negative instances are not considered. 
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If a definable family embeds in a maximum class, it inherits some good prop- 
erties, such as the existence of a compression scheme. In dimension 1 in any 
(weakly) o-minimal or strongly minimal theory, all definable families are sub- 
families of maximum definable families. We may ask how many of the good 
properties of such theories are related to this fact. 
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